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Abstract 

This paper describes procedures implemented in evaluating minority 
medical education programs operated by the University of Virginia School of 
Medicine since 1984 • Both process and outcome evaluation procedures have been 
used. Results of evaluation efforts are reviewed by program staff to design 
program improvements and modifications. 

Methods used in process evaluation have included: mid- and end-of- 
prograo interviews with program participants and program personnel; 
observation of program activities by the evaluator; informal contacts by the 
evaluator with students and program staff throughout the program, to allow 
ongoing feedback about program operations. Program outcomes are assessed 
primarily through two means: student scores on knowledge tests, and students 1 
educational and career choices following participation in the program. A 
computerized student tracking system has been developed to document post- 
program outcomes. 
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Using Evaluation Findings to Improve Minority Education Programs 

Since 1984, the University of Virginia School of Medicine has operated a 
variety of educational programs for minority students at the high school, 
undergraduate, and medical school levels. Throughout the course of these 
programs, evaluators have monitored program processes and assessed program 
outcomes. Evaluation findings are regularly reported to program 
administrators, who use the information in making decisions about program 
change and improvement. The purpose of this roundtable presentation is to 
illustrate the methods of process and outcome evaluation used by the School of 
Medicine's minority education programs over the course of several years, and 
to demonstrate how evaluation data have actually been used in ongoing program 
development. The report will focus on the past two years, during which the 
author served as evaluator for the programs. 

The University of Virginia School of Medicine's minority education 
programs are subsumed under the Assisting Students Achieve Medical Degrees 
(ASAMD) Project. The primary purpose of this project is to enhance the 
opportunities for minority and disadvantaged students to enter, remain in, and 
graduate from medical and health professions schools. The ASAMD Project is 
funded by the Health Careers Opportunity Program (HCOP) of the U. S. 
Department of Health and Human Services 1 Division of Disadvantaged Assistance, 
Bureau of Health Professions. The major components of the ASAMD Project are 
summer programs for high school and undergraduate students; ongoing support 
programs are also offered for minority students enrolled in the University of 
Virginia School of Medicine. 

At the high school level, through the Summer High School Academic Reach- 
up Program (SHARP) , students take a 6-week schedule of enrichment courses in 
writing and contnuni cation skills, math, are! science. A lecture series gives 



ERLC 



4 



the students the opportunity to hear successful minority professionals talk 
about various educational and career options available to them. In addition, 
apprenticeship programs sponsored by NIH and the University of Virginia School 
of Medicine provide several highly motivated students with an 8 to 10-week 
supervised research experience. The major goals at the high school level, 
then, are to bolster students' basic academic skills and to motivate than to 
stay in school beyond the high school years. 

At the undergraduate level, program goals are more specifically oriented 
towards medical education. A 6-week residential summer program (the Medical 
Academic Advancement Program, or MAAP) provides advance exposure to the 
content, pace, and volume of the medical school curriculum. Two types of 
courses are offered: (1) "bridging" courses developed around concepts and 
principles which mediate between undergraduate and medical school basic 
science courses, and (2) brief sections of first-year OVA medical school 
courses, presented by School of Medicine faculty in the same format that they 
use with first-year medical students. Included in the MAAP is a lecture 
series which introduces students to clinical aspects of many areas of 
medicine. Students also receive training in study skills, time management, 
and stress management. In short, the MAAP provides students with academic 
enrichment, exposure to medical school, and a unique opportunity for self- 
assessment of their interests and abilities. 

The approach to evaluating the ASAMD programs has included both process 
and outcome assessment. During the conduct of the summer programs, the 
evaluator is intensively involved in monitoring program activities. To this 
end, she observes each of the SHARP and MAAP classes and speaks informally 
with program instructors and students. At the beginning of the third week of 
each program, the evaluator schedules group interviews with the students in 
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both the SHARP and the MAAP. Students are asked for feedback about their 
satisfaction with all aspects of the programs. The evaluator also interviews 
program instructors at this point in the program. The information gained 
through these interviews is summarized and reported to program administrators 
who may make mid-program changes on the basis of the findings, or may plan 
revisions in future program offerings. In the past, for example, schedule 
changes have been made mid-program as a direct result of these interviews. At 
the end of each program, students complete comprehensive questionnaires 
evaluating the programs. 

Program outcomes are assessed primarily through two means: student 
scores on knowledge tests, and students 1 educational and career choices 
following participation in the program. To be more specific, for the MAAP, 
students take a pre- and post-test of their knowledge in chemistry and 
quantitative relationships, and receive grades in each of the MAAP courses. 
In the high school program, students also receive course grades. These, 
however, are short-term indicators of program success; more important in 
evaluating the effectiveness of these programs is documenting post-program 
student outcomes. To this end, a computerized student tracking system is 
being developed. To date, we knew that approximately 75% of the students who 
have been through the MAAP have entered medical school or graduate school in 
biomedical research, and the retention rate of these students is nearly 10051$. 
The data on the high school students' entry into college are not yet complete. 
Future evaluation plans are to refine the student tracking system, and to 
begin to compare educational outcomes for minority students who have and have 
not participated in University of Virginia School of Medicine educational 
programs. 



